1. Field of the Invention
The present invention relates to a manipulator system, a method for controlling a manipulator, a master manipulator, a method for controlling a master manipulator, a slave manipulator, a method for controlling a slave manipulator, and a storage medium. More particularly, the present invention relates to a manipulator system, a method for controlling a manipulator, a master manipulator, a method for controlling a master manipulator, a slave manipulator, a method for controlling a slave manipulator, and a storage medium, which allow the slave manipulator to be remotely controlled in an easy and accurate fashion.
2. Description of the Related Art
FIG. 1 illustrates the outer appearance of a conventional medical operation manipulator system.
A slave manipulator 3L and a slave manipulator 3R are disposed on an operating table 1. The slave manipulator 3L and the slave manipulator 3R (hereinafter, they are generically denotes as slave manipulators 3 when it is not needed to distinguish them from each other) are remotely controlled by a master manipulator 8L and a master manipulator 8R, respectively, so that a percutaneous operation is performed on a patient 2 laid (on his/her back in the example shown in FIG. 1) on the operating table.
The slave manipulator 3L is disposed on the left side (when the operating table 1 is seen from above) of the operating table 1. The slave manipulator 3L has a tip part 4L disposed on an end thereof, wherein the tip part 4L includes a treating tool such as a forceps, a knife, a suturing tool, or an injection.
The slave manipulator 3R is disposed on the right side (when the operating table 1 is seen from above) of the operating table 1. The slave manipulator 3L has a tip part 4R disposed on an end thereof, wherein the tip part 4R includes a treating tool such as a forceps, a knife, a suturing tool, or an injection.
Furthermore, a camera unit 5 including a CCD camera 6 disposed on an end thereof for taking an image of the inside of the abdominal cavity of the patient 2 is disposed on the operating table 1.
A manipulator stage 7, on which the master manipulators 8L and 8R manipulated by the operator A are disposed, is placed at a location properly apart from the operating table 1.
The master manipulator 8L is disposed on the left side (when seen by the operator at the back of whom the manipulator stage is disposed) of the manipulator stage 7. On the end of the master manipulator 8L, there is disposed a handling part 9L that is held and manipulated by the left hand of the operator A.
The master manipulator 8R is disposed on the right side of the manipulator stage 7. On the end of the master manipulator 8R, there is disposed a handling part 9R that is held and manipulated by the right hand of the operator A.
A monitor 10 is disposed near the manipulator stage 7 so that the operator A can see an image displayed on the monitor 10 when the operator A manipulates the master manipulators 8 (handling part 9). An image taken by the CCD camera 6 of the camera unit 5 is displayed on the monitor 10.
The operator A stands between the manipulator stage 7 and the monitor 10 such that the manipulator stage 7 is located at the back of the operator A and performs a percutaneous operation on the patient 2 in such a manner that, while watching the tip part 4 of the slave manipulator 3 displayed on the monitor 10, the operator A three-dimensionally manipulates the handling part 9L of the master manipulator 8L by his/her left hand thereby moving the tip part 4L of the slave manipulator 3L in synchronization with the motion of the handling part 9L and three-dimensionally manipulates the handling part 9R of the master manipulator 8R by his/her right hand thereby moving the tip part 4R of the slave manipulator 3R in synchronization with the motion of the handling part 9R.
Thus, in this system, one set of slave manipulators 3 (slave manipulators 3L and 3R) is remotely controlled by one set of master manipulators 8 (master manipulators 8L and 8R) so as to perform the percutaneous operation on the patient 2.
For example, when it is desirable to move the tip part 4 of the slave manipulator 3 along a curved path from a position A to a position B as shown in FIG. 2A (hereinafter, such a desirable path will be referred to as a target path), the operator A tries to move the handling part 9 of the master manipulator 8 along a target path corresponding the target path of the tip part 4 of the salve manipulator 3.
However, in practice, depending upon the degree of the skill of the operator A who manipulates the handling part 9, the actual path of the handling part 9 can deviate, as represented by a solid line in FIG. 2B, from the target path represented by a dotted line. The deviation of the actual path of the handling part 9 from the target path causes the path of the tip part 4 of the slave manipulator 3 to deviate from its target path.
As described above, in the system in which one slave manipulator 3 is remotely controlled by one master manipulator 8, the accuracy of the remote control greatly depends on the skill of the operator A.
That is, in the conventional system, in order to achieve high accuracy in the remote control of the slave manipulator 3, the operator A has to have great skill to operate the master manipulator 8 (handling part 9). In other words, it is not easy to achieve high accuracy in the remote control of the slave manipulator 3.
In view of the above, it is an object of the present invention to provide a technique of remotely controlling a slave manipulator using a plurality of master manipulators 8 thereby achieving high accuracy and ease in the remote control of the slave manipulator.
According to an aspect of the present invention, there is provided a manipulator system including a first master manipulator, a second master manipulator, and a slave manipulator, wherein the first master manipulator includes first detection means for detecting an absolute attitude, in a space within which the first handling part is allowed to move, of a first handling part of the first master manipulator, and first transmission means for transmitting the absolute attitude of the first handling part, the second master manipulator includes second detection means for detecting an absolute attitude, in a space within which the second handling part is allowed to move, of a second handling part of the second master manipulator, and second transmission means for transmitting the absolute attitude of the second handling part, and the slave manipulator includes first acquisition means for acquiring the absolute attitude of the first handling part transmitted from the first transmission means of the first master manipulator, and the absolute attitude of the second handling part transmitted from the second transmission means of the first master manipulator, first consolidation means for consolidating the absolute attitude of the first handling part and the absolute attitude of the second handling part, and first control means for controlling the attitude of the treating part in accordance with the result of the consolidation performed by the first consolidation means.
Preferably, the absolute attitude of the first handling part is the position and the state, of the first handling part, in a space within which the first handling part is allowed to move, and the absolute attitude of the second handling part is the position and the state, of the second handling part, in a space within which the second handling part is allowed to move.
Preferably, the first consolidation means of the slave manipulator calculates the weighted sum of the absolute attitude of the first handling part and the absolute attitude of the second handling part, using predetermined weighting factors for respective terms.
Preferably, the slave manipulator further includes third detection means for detecting a first force or a first torque applied to the treating part from the object being treated, and third transmission means for transmitting the first force or the first torque, the first master manipulator further includes second acquisition means for acquiring the first force or the first torque transmitted from the third transmission means of the slave manipulator, first determination means for determining a second force or a second torque to be perceived by the first operator, in accordance with the first force or the first torque acquired by the second acquisition means, and second control means for controlling the first handling part so that the first operator perceives the second force or the second torque, and the second master manipulator further includes third acquisition means for acquiring the first force or the first torque transmitted from the third transmission means of the slave manipulator, second determination means for determining a third force or a third torque to be perceived by the second operator, in accordance with the first force or the first torque acquired by the third acquisition means, and third control means for controlling the second handling part so that the second operator perceives the third force or the third torque.
Preferably, the slave manipulator further includes third transmission means for transmitting the result of the consolidation performed by the first consolidation means, the first master manipulator further includes second acquisition means for acquiring the result of the consolidation transmitted from the third transmission means of the slave manipulator, first calculation means for calculating the difference between the absolute attitude of the first handling part and the result of the consolidation acquired by the second acquisition means, first determination means for determining a force or a torque to be perceived by the first operator, in accordance with the result of the calculation performed by the first calculation means, and second control means for controlling the first handling part so that the first operator perceives the force or the torque determined by the first determination means, and the second master manipulator further includes second acquisition means for acquiring the result of the consolidation transmitted from the third transmission means of the slave manipulator, second calculation means for calculating the difference between the absolute attitude of the second handling part and the result of the consolidation acquired by the third acquisition means, second determination means for determining a force or a torque to be perceived by the second operator, in accordance with the result of the calculation performed by the second calculation means, and third control means for controlling the second handling part so that the second operator perceives the force or the torque determined by the second determination means.
Preferably, the slave manipulator further includes third detection means for detecting a first force or a first torque applied to the treating part from the object being treated, and third transmission means for transmitting the first force or the first torque and the result of the consolidation performed by the first consolidation means, the first master manipulator further includes second acquisition means for acquiring the first force or the first torque and the result of the consolidation transmitted from the third transmission means of the slave manipulator, first calculation means for calculating the difference between the absolute attitude of the first handling part and the result of the consolidation acquired by the second acquisition means, first determination means for determining a second force or a second torque to be perceived by the first operator, in accordance with the first force or the first torque acquired by the second acquisition means and in accordance with the difference calculated by the first calculation means, and second control means for controlling the first handling part so that the first operator perceives the second force or the second torque, and the second master manipulator further includes third acquisition means for acquiring the first force or the first torque and the result of the consolidation transmitted from the third transmission means of the slave manipulator, second calculation means for calculating the difference between the absolute attitude of the second handling part and the result of the consolidation acquired by the third acquisition means, second determination means for determining a third force or a third torque to be perceived by the second operator, in accordance with the first force or the first torque acquired by the third acquisition means and in accordance with the difference calculated by the second calculation means, and third control means for controlling the second handling part so that the second operator perceives the third force or the third torque.
The first master manipulator may further include first display control means for controlling a cue so that the first operator can operate the first handling part in synchronization with the operation of the second operator on the second handling part in accordance with the cue, and the second master manipulator may further include second display control means for controlling the cue so that the second operator can operate the second handling part in synchronization with the operation of the first operator on the first handling part in accordance with the cue.
The first master manipulator may further include first output control means for controlling a sound/voice cue so that the first operator can operate the first handling part in synchronization with the operation of the second operator on the second handling part in accordance with the sound/voice cue, and the second master manipulator may further include second output control means for controlling the sound/voice cue so that the second operator can operate the second handling part in synchronization with the operation of the first operator on the first handling part in accordance with the sound/voice cue.
Preferably, the system further includes a third master manipulator including a third handling part handled by a third operator, wherein the third master manipulator further includes third detection means for detecting a relative attitude of the third handling part with respect to a predetermined reference attitude, and third transmission means for transmitting the relative attitude of the third handling part detected by the third detection means, and wherein the first acquisition means of the slave manipulator further acquires the relative attitude of the third handling part transmitted from the third transmission means of the third master manipulator, and the first consolidation means consolidates the absolute attitude of the first handling part, the absolute attitude of the second handling part, and the relative attitude of the third handling part.
The first consolidation means of the slave manipulator may calculate the weighted sum of the absolute attitude of the first handling part, the absolute attitude of the second handling part, and the relative attitude of the third handling part, using predetermined weighting factors for respective terms.
The slave manipulator may further include fourth detection means for detecting a first force or a first torque applied to the treating part from the object being treated, and fourth transmission means for transmitting the first force or the first torque, the first master manipulator may further include second acquisition means for acquiring the first force or the first torque transmitted from the fourth transmission means of the slave manipulator, third acquisition means for acquiring the absolute attitude of the second handling part transmitted of the second transmission means of the second master manipulator, second consolidation means for consolidating the absolute attitude of the first handling part and the absolute attitude of the second handling part, first calculation means for calculating the difference between the absolute attitude of the first handling part and the result of the consolidation performed by the second consolidation means, first determination means for determining a second force or a second torque to be perceived by the first operator, in accordance with the first force or the first torque acquired by the second acquisition means and in accordance with the difference calculated by the first calculation means, and second control means for controlling the first handling part so that the first operator perceives the second force or the second torque, the second master manipulator may further include second acquisition means for acquiring the first force or the first torque transmitted from the fourth transmission means of the slave manipulator, fifth acquisition means for acquiring the absolute attitude of the first handling part transmitted of the first transmission means of the first master manipulator, third consolidation means for consolidating the absolute attitude of the second handling part and the absolute attitude of the first handling part, second calculation means for calculating the difference between the absolute attitude of the second handling part and the result of the consolidation performed by the third consolidation means, second determination means for determining a third force or a third torque to be perceived by the second operator, in accordance with the first force or the first torque acquired by the fourth acquisition means and in accordance with the difference calculated by the second calculation means, and third control means for controlling the second handling part so that the second operator perceives the third force or the third torque, and the third master manipulator may include sixth acquisition means for acquiring the first force or the first torque transmitted from the fourth transmission means of the slave manipulator, third determination means for determining a fourth force or a fourth torque to be perceived by the third operator, in accordance with the first force or the first torque acquired by the sixth acquisition means and in accordance with the relative attitude of the third handling part, and fourth control means for controlling the third handling part so that the third operator perceives the fourth force or the fourth torque.
The first master manipulator may further include seventh acquisition means for acquiring the relative attitude of the third handling part transmitted from the third transmission means of the third master manipulator, the first determination means may determine the second force or the second torque such that if a value corresponding to the relative attitude of the third handling part is smaller than a predetermined threshold value, the second force or the second torque is determined in accordance with the first force or the first torque and the result of the consolidation performed by the second consolidation means, however if the value corresponding to the relative attitude of the third handling part is equal to or greater than the predetermined threshold value, the second force or the second torque is determined in accordance with only the result of the consolidation performed by the second consolidation means, the second master manipulator may further include eighth acquisition means for acquiring the relative attitude of the third handling part transmitted from the third transmission means of the third master manipulator, the second determination means may determine the third force or the third torque such that if the value corresponding to the relative attitude of the third handling part is smaller than the predetermined threshold value, the third force or the third torque is determined in accordance with the first force or the first torque and the result of the consolidation performed by the third consolidation means, however if the value corresponding to the relative attitude of the third handling part is equal to or greater than the predetermined threshold value, the third force or the third torque is determined in accordance with only the result of the consolidation performed by the third consolidation means, and the third determination means of the third master manipulator may determine the fourth force or the fourth torque such that if the value corresponding to the relative attitude of the third handling part is smaller than the predetermined threshold value, the fourth force or the fourth torque is determined in accordance with only the relative attitude of the third handling part, however if the value corresponding to the relative attitude of the third handling part is equal to or greater than the predetermined threshold value, the fourth force or the fourth torque is determined in accordance with the relative attitude of the third handling part and the first force or the first torque.
According to another aspect of the present invention, there is provided a method of controlling a manipulator, including a first detection step of detecting an absolute attitude, in a space within which the first handling part is allowed to move, of the first handling part of the first master manipulator; a first transmission step of transmitting the absolute attitude of the first handling part; a second detection step of detecting an absolute attitude, in a space within which the second handling part is allowed to move, of the second handling part of the second master manipulator; a second transmission step of transmitting the absolute attitude of the second handling part; a first acquisition step of acquiring the absolute attitude of the first handling part transmitted in the first transmission step and the absolute attitude of the second handling part transmitted in the second transmission step; a first consolidation step of consolidating the absolute attitude of the first handling part and the absolute attitude of the second handling part; and a first control step of controlling the treating part in accordance with the result of the consolidation performed in the first consolidation step.
According to still another aspect of the present invention, there is provided a first storage medium in which is stored a program including a first detection step of detecting an absolute attitude, in a space within which the first handling part is allowed to move, of the first handling part of the first master manipulator; a first transmission step of transmitting the absolute attitude of the first handling part; a second detection step of detecting an absolute attitude, in a space within which the second handling part is allowed to move, of the second handling part of the second master manipulator; a second transmission step of transmitting the absolute attitude of the second handling part; a first acquisition step of acquiring the absolute attitude of the first handling part transmitted in the first transmission step and the absolute attitude of the second handling part transmitted in the second transmission step; a first consolidation step of consolidating the absolute attitude of the first handling part and the absolute attitude of the second handling part; and a first control step of controlling the treating part in accordance with the result of the consolidation performed in the first consolidation step.
In the above-described manipulator system, manipulator control method, and first storage medium in which the program is stored, the absolute attitude, in the space within which the first handling part is allowed to move, of the first handling part of the first master manipulator is detected, the absolute attitude of the first handling part is transmitted, the absolute attitude, in the space within which the second handling part is allowed to move, of the second handling part of the second master manipulator is detected, the absolute attitude of the second handling part is transmitted, the absolute attitude of the transmitted first handling part and the absolute attitude of the transmitted second handling part are acquired, the absolute attitude of the first handling part and the absolute attitude of the second handling part are consolidated, and the attitude of the treating part is controlled in accordance with the consolidation result.
According to still another aspect of the present invention, there is provided a master manipulator comprising detection means for detecting the attitude of a handling part; and transmission means for transmitting the attitude of the handling part to a slave manipulator thereby allowing the slave manipulator to control a treating part of the slave manipulator so as to process an object in accordance with a result of consolidation of the attitude of the handling part and the attitude of a handling part of another master manipulator supplied from said another master manipulator.
According to still another aspect of the present invention, there is provided a method for controlling a master manipulator, the method comprising a detection step for detecting the attitude of a handling part; and a transmission step for transmitting the attitude of the handling part to a slave manipulator thereby allowing the slave manipulator to control a treating part of the slave manipulator so as to process an object in accordance with a result of consolidation of the attitude of the handling part and the attitude of a handling part of another master manipulator supplied from said another master manipulator.
According to still another aspect of the present invention, there is provided a second storage medium in which is stored a program comprising a detection step for detecting the attitude of a handling part; and a transmission step for transmitting the attitude of the handling part to a slave manipulator thereby allowing the slave manipulator to control a treating part of the slave manipulator so as to process an object in accordance with a result of consolidation of the attitude of the handling part and the attitude of a handling part of another master manipulator supplied from said another master manipulator.
In the above-described master manipulator, the method for controlling the master manipulator, and the second storage medium in which the program is stored, the attitude of the handling part is detected, the detected attitude of the handling part is transmitted to the slave manipulator, and the slave manipulator is controlled in accordance with the consolidation of the attitude of the handling part and the attitude of the handling part of another master manipulator supplied from said another master manipulator so that the treating part of the slave manipulator correctly processes the object.
According to still another aspect of the present invention, there is provided a slave manipulator comprising acquisition means for acquiring the attitude of a first handling part of a first master manipulator transmitted from the first master manipulator and the attitude of a second handling part of a second master manipulator transmitted from the second master manipulator; consolidation means for consolidating the attitude of the first handling part and the attitude of the second handling part; and control means for controlling the attitude of the treating part in accordance with the result of the consolidation performed by the consolidation means.
According to still another aspect of the present invention, there is provided a method for controlling a slave manipulator, comprising an acquisition step for acquiring the attitude of a first handling part of a first master manipulator transmitted from the first master manipulator and the attitude of a second handling part of a second master manipulator transmitted from the second master manipulator; a consolidation step for consolidating the attitude of the first handling part and the attitude of the second handling part; and a control step for controlling the attitude of the treating part in accordance with the result of the consolidation performed in the consolidation step.
According to still another aspect of the present invention, there is provided a third storage medium in which is stored a program comprising an acquisition step for acquiring the attitude of a first handling part of a first master manipulator transmitted from the first master manipulator and the attitude of a second handling part of a second master manipulator transmitted from the second master manipulator; a consolidation step for consolidating the attitude of the first handling part and the attitude of the second handling part; and a control step for controlling the attitude of the treating part in accordance with the result of the consolidation performed in the consolidation step.
In the above-described slave manipulator, method for controlling the slave manipulator, and third storage medium in which the program is stored, the attitude of the first handling part of the first master manipulator transmitted from the first master manipulator and the attitude of the second handling part of the second master manipulator transmitted from the second master manipulator are acquired, the attitudes of the first and second handling parts are consolidated, and the attitude of the treating part is controlled in accordance with the consolidation result.